Thus far, there has been known a method in which linear light being applied to a surface of an object to be measured is imaged and the surface shape of the object to be measured is measured on the basis of what is called a light-section method; for example, Patent Literature 1 below proposes a method in which linear laser light applied along the width direction of an object to be measured such as a slab, which is a steel semi-finished product in the course of conveyance, is successively imaged by an area camera, a luminance image and an unevenness image are generated from the obtained captured image, and then the surface of the object to be measured is inspected on the basis of the luminance image and the unevenness image. In this method, the principle of the light-section method is applied to the plurality of successively captured images (light-section images) to generate an unevenness image, and a defect having an unevenness change of the surface, such as a vertical crack having an opening, is detected on the basis of the unevenness image. Further, in this method, a luminance image is generated from information on the brightness (that is, luminance) of the light-section line, and is used to detect a defect having a reflectance change that does not have an opening, such as a transverse crack.